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(54) Methods and apparatus for using metadata from different sources 



(57) Metadata from different sources are used to or- 
ganize recordings, search for content and access spe- 
cific recordings. Due to different metadata sources us- 
ing different standards, the metadata have different se- 
mantics and syntax. According to the invention, from the 
original metadata received from the different sources 



unified metadata are derived using one or more trans- 
formation rules. The transformation rules specify how 
the content or parts of the content of said original meta- 
data shall be used to constitute the content or part of 
the content of said unified metadata. The derived unified 
metadata are presented to a user and processed in re- 
sponse to a user input. 
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Description 

[0001] The invention relates to a method and to an 
apparatus for using metadata from different sources, es- 
pecially for organizing of recordings, searching for con- 
tent and accessing specific recordings. 

Background 

[0002] The capacity of digital storage media for per- 
sonal video recording grows continuously. Currently it is 
possible to store about 20 full-length movies on a single 
1 00 gigabyte (GB) hard disk. In 2005, it will likely be pos- 
sible to store about 80 movies on a single 400 GB hard 
disk. 

[0003] Similarly, the capacity of exchangable storage 
media is expanding. DVDs containing full-length movies 
are widely available and also rewritable DVDs 
(DVD-RAM, DVD-RW, DVD+RW) are on the market, 
having a storage capacity of 4.7 GB. Furthermore, the 
basic specification for a next generation of large capac- 
ity optical discs called "Blu-ray Disc" has recently been 
established, enabling the recording, rewriting and play- 
back of up to 27 GB of data on a single sided single layer 
disc. In order to further increase the capacity, two or 
more layers may be used per side and these may be 
applied to both sides of the disc. Finally, several discs 
may be combined in a special magazine. 
[0004] This enormous amount of data requires new 
ways to organize the recordings, search for content and 
access specific recordings. One possible solution for 
this is to use so-called metadata, defined as data about 
data, for the recorded content. Various industry groups 
and standard bodies have been developing metadata 
standards for different purposes and applrcatlons. In the 
context of metadata usage, the content to which meta- 
data are related is often called "essence". 
[0005] The compression standards for AV streams 
MPEGrl. MPEG-.2 and MPEG-4 already contain some 
program specific information (PSI), specified in the re- 
spective MPEG systems standard. Similarly, the DVB 
standard used for the transmission of digital television 
signalis specifies Service Information (DVB-SI) included 
in a DVB compliant MPEG-2 transport stream multiplex. 
[0006] More recently, the TV-Anytime Forum defined 
In the specification S-3 on Metadata, document number 
SP003v1 .1 metadata which allow the consumer to find, 
navigate and manage content from a variety of internal 
and extemal sources Including, for example, enhanced 
broadcast, interactive TV, Internet and local storage. 
Additionally, these metadata may also include informa- 
tion about user preferences, such as favorite actors or 
TV shows, which facilitate automatic filtering and acqui- 
sition of content by agents on behalf of the consumer. 
[0007] Similarly, an MPEG standard for the effective 
and efficient access to multimedia content is currently 
underdevelopment [ISO/I EC 15938: Information Tech- 
nology ~ Multimedia content description Interface]. This 



2 

MPEG-7 standard will offer metadata elements and their 
structure and relationships, that are defined in the form 
of Descriptors and Description Schemes to create de- 
scriptions of the multimedia content. 

5 

Invention 

[0008] The invention is based on the recognition of the 
following fact. Often several different metadata describ- 

10 ing the same content may be available. However, these 
metadata may have different associated meanings (se- 
mantics) and ways of coding (syntax), which are defined 
in the respective standards, e.g. the MPEG-7 or TV-An- 
ytime standard. On the other hand, in systems planning 

IS to employ metadata, certain semantics and syntax of 
metadata or metadata fields will be more appropriate 
than others, depending on the properties and features 
the system shall be able to perform. Therefore, it is de- 
sirable to allow the use of all available metadata, despite 

20 their different semantics and syntax. 

[0009] Therefore, a problem to be solved by the in- 
vention is to allow the use of metadata originating from 
different sources having different semantics or syntax. 
[0010] This problem is solved by the method dis- 

25 closed in claim 1 . An apparatus that utilizes this method 
is disclosed in claim 8. 

[0011] According to the invention original metadata 
are received from different sources and unified metada- 
ta are derived from said original metadata using one or 

30 more transformation rules, wherein the transformation 
rules specify how the content or parts of the content of 
said original metadata shall be used to constitute the 
content or part of the content of said unified metadata. 
[0012] Advantageously, the derived unified metadata 

35 are presented to a user and are processed in response 
to a user input. 

[0013] Advantageously, the processing results in se- 
lecting, deleting or modifying said unified metadata. 
[0014] Furthermore, it is of advantage to store the 
original metadata, keep the stored original metadata un- 
changed and enable to recurrfrom the unified metadata 
to the original metadata. 

[0015] For the transfonmatlon rules, it is advanta- 
geous to base them on fonmal and/or structural proper- 

^5 ties of the original metadata in all cases where the orig- 
inal metadata are themself structured. 
[0016] For unstructured original metadata of textual 
nature, it Is advantageous to base the transformation 
rules on a similarity measure which allows for a tolerant 

50 matching of phrases against a user-defined search 
phrase. 

[0017] According to a further advantageous embodi- 
ment a part of the transformation rules covers the map- 
ping or inheritance of the essence links from within the 
55 original metadata into ail derived unified metadata. 
[001 8] Further advantageous embodiments of the in- 
vention result from the following description. 
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Drawing 

[0019] Exemplary embodiments of the invention are 
described with reference to the accompanying draw- 
ings, which show in: 

Fig. 1 generically the conversion of original metadata 
records with different syntax and semantics in- 
to unified metadata records; 

Fig.2. a block diagram of the processing of metadata 
Including the derivation of unified metadata 
from received original metadata. 

Exemplary embodiments 

[0020] Figure 1 shows generically the conversion of 
original metadata records with different syntax and se- 
mantics into unified metadata records, in the following 

also called Core Metadata. Transfonnations T1 Tj,... 

specify rules how the content or parts of the content of 
the original metadata records of a predetermined class 
shall be used to constitute the content or part of the con- 
tent of a unified metadata record. In Figure 1 , the blocks 
"ODRI" to "ODRk+m" symbolize different classes of 
original metadata records, and more specifically, It shall 
be assumed that blocks ODR1 and ODRk both are gov- 
erned by one specific syntax/semantics, whereas 
blocks ODRk+1 and ODRk+m are govemed by a sec- 
ond, different syntax/semantics. All original metadata 
records of class ODR1 are converted by a set of trans- 
formations T1 into instances of a first class of unified 
metadata records UDR1 . As part of this conversion, the 
transformation T1 may mandate the presence of certain 
parts within the instance of ODR1 , so that by applying 
T1 , not all Instances of ODR1 will actually generate an 
output UDR1 . Also, it must be kept in mind, that the con- 
version described by T1 may mandate that only certain 
parts of the instance of ODR1 shall be taken to consti- 
tute certain pads of the emerging instance of UOR1. 
Similarly, the same set of transformations T1 may con- 
tain a rule which defines that, under certain conditions, 
some predefined parts of instances of a second class of 
original metadata records ODRk are converted into oth- 
er instances of the first kind of unified metadata records 
UDR1\ However, other instances of the second class of 
original metadata ODRk. if they contain certain other 
parts, are converted by a different set of transfonnations 
Tj into instances of a second class of unified metadata 
records UDRj. Similariy. further original metadata 
records of classes ODRk+1 and ODRk+m having a syn- 
tax/semantics according to a second standard are con- 
verted by the transformations T1 and Tj into Instances 
of the first and second unified metadata records UDR1 
and UDRj. In this way, when a sufficient number of orig- 
inal metadata is available and the respective transfor- 
mation rules are applied, several candidates will be gen- 
erated for each kind of Core Metadata. In this way, a 
user has sufficient material from which he can editorially 
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distill the Core Metadata he finally wants. 
[0021 ] Figure 2 shows an exarfiple for the derivation 
of Core Metadata from originally received metadata as 
part of an overall metadata processing chain. Although 
s the further description concentrates on the processing 
of an MPEGi-2 transport stream, it can easily be gener- 
alised for use in any system using multiplexed bit- 
streams comprising metadata. 

[0022] When recording a broadcast DVB transport 

10 stream TS additional service information encompassed 
in the multiplex besides the audio and video signals is 
parsed from the multiplex. For this purpose the broad- 
cast signal is analyzed for metadata like teletext, 
DVB-SI etc. by a DVB metadata parser MD-PAR and 

IS the detected metadata are demultiplexed and extracted. 
[0023] The extracted metadata are transfomned into 
XML metadata descriptors XML-DS. This XML repre- 
sentation for the metadata descriptors allows to have a 
generic framework for different kinds of metadata de- 

^ scriptors. In other words, the metadata database 
MD-DB can store any kind of metadata as long as there 
is a transformer that converts it from its native represen- 
tation into the XML representation. Similariy it is possi- 
ble to perform queries on the metadata, independent of 

55 their respective metadata sources. 

[0024] There are different transfomners for different 
metadata sources, like e.g. a transformer XML-Sl for the 
transformation of DVB-SI data or a further transformer 
XML-TT for the transfomiation of Teletext or Closed 

30 Caption data. The transformers transform the metadata 
from their native coding, e.g. DVB-SI binary fomiat, into 
an XML representation. This transformation is keeping 
the information inside the metadata "as Is". It neither 
adds nor removes informatbn. This allows a very flexi- 

35 ble later use of the metadata. During recording the meta- 
data descriptors are accumulated and stored into the 
metadata database MD-DB. 

[0025] Similarly, metadata descriptors generated 
manually by the user such as user-annotations UA may 

40 be stored after they have been transformed by a trans- 
former XML-UA into XML representation. Furthermore, 
metadata descriptors according to other standards like 
HTML, TV-Anytime, MPEG-7 or SMPTE are trans- 
formed by respective transfomners XML-HTML, 

45 XML-TVA. XML-M7 or XML-SMPTE and stored. These 
metadata descriptors may be received via the same or 
other transmission paths like the Internet. 
[0026] An archive system may also store metadata 
descriptors originating from other recording devices 

50 and/or storage media. However, this necessitates an ex- 
port and import facility MD-EXP-IMP for metadata de- 
scriptors, such that metadata descriptors can be ex- 
changed between devk:es and/or media. 
[0027] In order to use the metadata descriptors stored 

55 in the metadata database MD-DB the inventive conver- 
sion of the original metadata into Core Metadata CM has 
to be performed using specific transformation rules 
MD-T. Here is where it becomes beneficial that the 
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metadata is stored "as is". No information has been add- 
ed or removed from the complete set of metadata infor- 
mation that is available. 

[0028] Finally, access may be provided to all metada- 
ta via a user interface Ul with query and browsing tech- 
niques Q for finding the corresponding video/audio 
stream or metadata the selected metadata descriptor is 
pointing to. 

[0029] One example of the conversion of the original 
metadata into Core Metadata is the conversion of differ- 
ent metadata items into title information for a given re- 
cording. Often the title information is the most useful and 
versatile single metadata item when it comes to archiv- 
ing, searching and sorting of any kind of AV material. 
For all material, be it self recorded (e.g. with a camcord- 
er), or recorded from broadcast, or be it on canned me- 
dia, mostly everybody will be able to associate a Title 
with it, under which the material can, on one hand, be 
recognized for later search, but, on the other hand, can 
also be sufficiently differentiated. If necessary, against 
other material which might be closely similar. 
[0030] Con'esponding to this very intuitive notion of 
Titfe, many existing metadata systems comprise items 
which constitute a container for Titles, although they 
may come under different names and have distinct syn- 
taxes. 

DVB, in its "DVB-SI" system of Service Infomnation, 
defines a "short event descriptor^ which 'provides 
the name of the event and a short description of the 
event in text form". The syntax of the short event 
descriptor foresees two variable length text fields: 
a dedicated one for the 'event_name'', and another 
one to 'specify the text description of the event'. 
MPEG-7, in its 'Creation Description Scheme', has 
the notion of 'Title", as the name of an audio-visual 
program". 

TVAnytime, in its work on Electronic Program 
Guides, mentions the concepts of "Title" as 'T&ctual 
title of the program (...) Multiple title descriptors may 
t>e included". Similar concepts of 'Episode Title' 
and 'Series Title' exist. 

Any container for any kind of User Annotations 
might always happen to be used for a user-specific 
Title of a piece of AV material. 

[0031] Conrespondingly, a Core Metadata scheme ac- 
cording to the present invention, will have one class of 
Core Metadata named 'Title', and all the above men- 
tioned given metdata should be used as Input to derive 
"77f/e" candidates, because - depending on what they 
contain - they might contain good hiaterial to be used 
for the above mentioned intuitive notion of "Tltld'. 
Hence, the Core Metadata scheme will define transfor- 
mation rules for each of the metadata above, specifying 
in each case which part of the content should be inserted 
as the content of the derived Core Metadata item. The 
Core metadata scheme may then provide tools such 
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that the user can inspect ail the 'Title" candidates, select 
one of them and/or edit their content as to match his/her 
personal preference. 

[0032] An even more illustrate example for the deri- 
5 vation of T/tfe'candidates could be as follows. ERG da- 
ta are transmitted according to theTV-Anytime standard 
and are received by a TV receiver or set top box. A user 
invokes a display of the EPG on a television screen and 
selects a television program in the displayed EPG based 

10 on the displayed title. The title information according to 
the TV-Anytime standard is separated and stored as a 
first candidate of the Core MD "title". After tuning the TV 
receiver to the selected channel, the Sl-infoimation 
within the DVB signal is analyzed and a second title in- 

15 formation is separated and stored as a second candi- 
date. Furthermore, the teletext signal received on the 
received channel is checked for information about the 
cun-ent program, which is stored as third title candidate. 
Since the stored title information is based on different 
20 standards, different transfoimations are applied to the 
title informations in order to convert them all into the 
same format. All converted title informations are stored, 
and at any convenient Jater time, they can be displayed 
to the user and the user then can select the "best" title 

25 information . For example, for a long title two of the three 
title infonnations may happen to be abbreviated ver- 
sions while only the third one may happen to be com- 
plete. The user may in this case select the complete title 
information for future overviews of his recordings. 

30 [0033] Metadata being defined as "data about data", 
a constituent part of any original metadata will always 
be an essence link, by which the metadata points to an 
AV essence. From this it is obvious, that parts of the 
transformation rules MD-T will advantageously describe 

35 the details how the essence links found on an input 
metadata shall be duplicated in order to appear on the 
derived Core Metadata, too. In addition to essence links, 
an original metadata may also contain "origin links" 
specifying where the original metadata was received or 

40 extracted from. Assuming that Core Metadata, too, may 
have such an "origin link", another part of the transfor- 
mation rule MD-T may describe how this Core Metadata 
origin link shall be set, one possible solution being that 
this link points to the original metadata from which the 

45 core metadata was generated. 

[0034] For original metadata which are themself 
structured, like those according to the DVB-SI or TV-An- 
ytime standard, the transfomiation rules MD-T will typi- 
cally select only certain kinds of. metadata for further 

50 processing, will test for the presence of certain optional 
structural parts of the input metadata, will potentially 
check some parts whether they have prescribed values, 
and, conditional on these preconditions being met, the 
transformation will extract a certain subset of the input 

55 metadata and embed it in a certain position of the output 
Core Metadata to be generated. 
[0035] For other input metadata, especially those 
which are unstructured except being indicated as "tex- 
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5. Method according to any of claims 1 to 4. wherein 
said original metadata contain links, and where a 
part of said transformation rules covers the map- 
ping or in heritance of said links from within said orig- 

s inal metadata Into all derived said unified metadata. 

6. Method according to any of claims 1 to 5, wherein 
said original metadata are at least partly structured 
and where said transformation rules are at least 

10 partly based on examining and evaluating the struc- 
ture of said original metadata. 



tuar, a transformation rule MD-T may typically consist 
in evaluating a phrase similarity measure between the 
input metadata and some user-provided search phrase. 
Such a phrase similarity measure should advanta- 
geously be made tolerant against slight variations of 
grammar or wording, which can be achieved by evalu- 
ating word order and word distance, and by additionally 
converting all words to be compared into word stems 
using algorithms known in the art as "stemming". 
[0036] The invention is especially useful for personal 
video recording, e.g. for the search and access of spe- 
cific recordings on high-volume storage media. Howev- 
er, the invention is also applicable to all other kinds of 
electronic multimedia content referencing and content 
location. The recorded content may be movies and tel- 
evision programs but also every other form of audiovis- 
ual infonnation. such as arbitrary combinations of still 
pictures, graphics, 3D models, audio, speech and video. 



Claims 

1. Method for using metadata from different sources, 
said metadata describing and being linked to multi- 
media essence and being used for accessing said 
multimedia essence, characterized by 

receiving original metadata from different 
sources; 

deriving from said original metadata unified 
metadata using one or more transfomnation 
rules, said transformation rules specifying how 
the content or parts of the content of said orig- 
inal metadata shall be used to constitute the 
content or part of the content of said unified 
metadata. 

2. Method according to claim 1 , further characterized 

by 

presenting said derived unified metadata to a 
usen 

processing said derived unified metadata in re- 
sponse to a user input. 

3. Method according to claim 2, wherein said process- 
ing results in selecting, deleting or modifying said 
unified metadata. 

4. Method according to any of claims 1 to 3, further 
characterized by 

storing the original metadata; 
keeping the stored original metadata un- 
changed; and 

enabling to recurr from the unified metadata to 
the original metadata. 



7. Method according to any of claims 1 to 5, wherein 
said original metadata at least partly consist of se- 

is quences of words not othenwise structured, where 
said transformation at least partly involves a search 
phrase, where said transformation rules are at least 
partly based on evaluating a phrase similarity 
measure, where said phrase similarity measure is 

20 based on word order and/or word distance meas- 
ures, and where, prior to applying said phrase sim- 
ilarity measure, the words of said original metadata 
and of said search jph rase are optionally normalized 
by applying a stemming algorithm. 

25 

8. Apparatus for using metadata from different sourc- 
es, said metadata describing multimedia essence 
and being used for accessing said multimedia es- 
sence, characterized In 

30 

means for receiving original metadata from dif- 
ferent sources; 

means for deriving from said original metadata 
unified metadata using one or more transfor- 
35 mation rules, said transformation rules specify- 

ing how the content or parts of the content of 
said original metadata shall be used to consti- 
tute the content or part of the content of said 
unified metadata. 

40 

9. Apparatus according to claim 8, further character- 
ized by 

means for presenting said derived unified 
45 metadata to a user; 

means for processing said derived unified 
metadata in response to a user input. 
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